How to Fine-Tune BERT for Text Classification?

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets

Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition

Emotion Recognition in Conversation (ERC) has been widely studied due to its importance in developing emotion-aware empathetic machines. The rise of pre-trained language models (PLMs) has further pushed the limit of ERC performance. However, most recent works on ERC using PLMs are heavily data-driven, and requires fine-tuning the entire PLMs. To improve both sample and computational efficiency, we propose a derivative-free optimization method called Cross-Task Prompt Tuning (CTPT) for few-shot conversational emotion recognition. Unlike existing methods that learn independent knowledge from individual tasks, CTPT leverages sharable cross-task knowledge by exploiting external knowledge from other source tasks to improve learning performance under the few-shot setting. Moreover, CTPT only needs to optimize a vector under the low intrinsic dimensionality without gradient, which is highly parameter-efficient compared with existing approaches. Experiments on five different contextual conversation datasets demonstrate that our CTPT method has superior results on both few-shot scenarios and zero-shot transfers.Comment: Findings of EMNLP 202

Towards Generalizable Graph Contrastive Learning: An Information Theory Perspective

Graph contrastive learning (GCL) emerges as the most representative approach for graph representation learning, which leverages the principle of maximizing mutual information (InfoMax) to learn node representations applied in downstream tasks. To explore better generalization from GCL to downstream tasks, previous methods heuristically define data augmentation or pretext tasks. However, the generalization ability of GCL and its theoretical principle are still less reported. In this paper, we first propose a metric named GCL-GE for GCL generalization ability. Considering the intractability of the metric due to the agnostic downstream task, we theoretically prove a mutual information upper bound for it from an information-theoretic perspective. Guided by the bound, we design a GCL framework named InfoAdv with enhanced generalization ability, which jointly optimizes the generalization metric and InfoMax to strike the right balance between pretext task fitting and the generalization ability on downstream tasks. We empirically validate our theoretical findings on a number of representative benchmarks, and experimental results demonstrate that our model achieves state-of-the-art performance.Comment: 25 pages, 7 figures, 6 table

Inhibitory mechanism of vortioxetine on CYP450 enzymes in human and rat liver microsomes

Vortioxetine is a novel anti-major depression disorder drug with a high safety profile compared with other similar drugs. However, little research has been done on drug-drug interactions (DDI) about vortioxetine. In this paper, the inhibitory effect of vortioxetine on cytochrome P450 (CYP450) and the type of inhibitory mechanism were investigated in human and rat liver microsomes. We set up an in vitro incubation system of 200 μL to measure the metabolism of probe substrates at the present of vortioxetine at 37°C. The concentrations of the metabolites of probe substrates were all measured by ultra-performance liquid chromatography tandem mass spectrometry (UPLC-MS/MS) method. It was found no time-dependent inhibition (TDI) of vortioxetine through determination of half-maximal inhibitory concentration (IC50) shift values. The enzymes and metabolites involved in this experiment in human and rats were as follows: CYP3A4/CYP3A (midazolam); CYP2B6/CYP2B (bupropion); CYP2D6/CYP2D (dextromethorphan); CYP2C8/CYP2C-1 (amodiaquine); CYP2C9/CYP2C-2 (losartan); and CYP2C19/CYP2C-3 (mephenytoin). We found that vortioxetine competitively inhibited CYP2C19 and CYP2D6 in human liver microsomes (HLMs) with inhibition constant (Ki) values of 2.17 μM and 9.37 μM, respectively. It was noncompetitive inhibition for CYP3A4 and CYP2C8, and its Ki values were 7.26 μM and 6.96 μM, respectively. For CYP2B6 and CYP2C9, vortioxetine exhibited the mixed inhibition with Ki values were 8.55 μM and 4.17 μM, respectively. In RLMs, the type of vortioxetine inhibition was uncompetitive for CYP3A and CYP2D (Ki = 4.41 and 100.9 μM). The inhibition type was competitive inhibition, including CYP2B and CYP2C-2 (Ki = 2.87 and 0.12 μM). The inhibition types of CYP2C-1 and CYP2C-3 (Ki = 39.91 and 4.23 μM) were mixed inhibition and noncompetitive inhibition, respectively. The study of the above mechanism will provide guidance for the safe clinical use of vortioxetine so that the occurrence of DDI can be avoided

Elemental topological Dirac semimetal: {\alpha}-Sn on InSb(111)

Three-dimensional (3D) topological Dirac semimetals (TDSs) are rare but important as a versatile platform for exploring exotic electronic properties and topological phase transitions. A quintessential feature of TDSs is 3D Dirac fermions associated with bulk electronic states near the Fermi level. Using angle-resolved photoemission spectroscopy (ARPES), we have observed such bulk Dirac cones in epitaxially-grown {\alpha}-Sn films on InSb(111), the first such TDS system realized in an elemental form. First-principles calculations confirm that epitaxial strain is key to the formation of the TDS phase. A phase diagram is established that connects the 3D TDS phase through a singular point of a zero-gap semimetal phase to a topological insulator (TI) phase. The nature of the Dirac cone crosses over from 3D to 2D as the film thickness is reduced

Research on the Internationalization Development and Cooperation Path of Higher Education Between Turkey and China

In recent years, with the irreversible trend of internationalization worldwide, the internationalization of higher education is a key direction of China’s education in the future. Given the current situation of political, economic, and social development in Balkan countries, Turkey strongly desires to develop its internationalization of higher education. This paper will explore the internationalization development status of higher education between Turkey and China, based on which their cooperation path of higher education is summarized

The Existence of Solutions for Boundary Value Problem of Fractional Functional Differential Equations with Delay

A class of boundary value problem for fractional functional differential equation with delay C D σ ω ( t ) = f ( t , ω t ) , t ∈ [ 0 , ζ ] , ω ( 0 ) = 0 , ω ′ ( 0 ) = 0 , ω ″ ( ζ ) = 1 , \left\{ {\begin{array}{*{20}c} {^{C} D^{\sigma } \omega (t) = f(t,\omega _{t} ),t \in [0,\zeta ],} \\ {\omega (0) = 0,\,\omega ^{\prime}(0) = 0,\,\omega ^{\prime\prime}(\zeta ) = 1,} \\ \end{array} } \right. is studied, where 2 < σ ≤ 3 , c D σ 2 < \sigma \le 3,\,\,^{c} D^{\sigma } devote standard Caputo fractional derivative. In this article, three new criteria on existence and uniqueness of solution are obtained by Banach contraction mapping principle, Schauder fixed point theorem and nonlinear alternative theorem